Chaotic properties of systems with Markov dynamics 
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We present a general approach for computing the dynamic partition function of a continuous- 
time Markov process. The Ruelle topological pressure is identified with the large deviation function 
of a physical observable. We construct for the first time a corresponding finite Kolmogorov- Sinai 
entropy for these processes. Then, as an example, the latter is computed for a symmetric exclusion 
process. We further present the first exact calculation of the topological pressure for an ./V-body 
stochastic interacting system, namely an infinite-range Ising model endowed with spin-flip dynamics. 
Expressions for the Kolmogorov-Sinai and the topological entropies follow. 
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In statistical mechanics, bridging the microscopies to 
the macroscopics remains the ultimate goal, be it in or 
out of equilibrium. The development of the theory of dy- 
namical systems and of their chaotic properties has led to 
major advances in equilibrium and nonequilibrium statis- 
tical mechanics. All those approaches make extensive use 
of such concepts as Lyapunov exponents, Kolmogorov- 
Sinai (KS) or topological entropies, topological pressure, 
etc., all quite mathematical in nature, and for which very 
few results (even nonrigorous) are available, as far as sys- 
tems with many degrees of freedom are concerned. One of 
the central ideas in constructing a statistical physics out 
of equilibrium is that of Gibbs ensembles Q in which time 
is seen to play the role of the volume in traditional equi- 
librium statistical mechanics. A central quantity called 
the dynamical partition function is in general defined as 



Z(s,t)= (ProMhistory})^ 



(1) 



histories from — >t 



The so-called thermodynamic formalism allows to derive 
from this quantity a number of chaotic properties such 
as the topological entropy which is defined through the 
number of possible trajectories a system can follow, or the 
KS entropy which is a measure of the complexity of a pro- 
cess and characterizes its dynamical randomness. Within 
that framework there exists a mathematical construc- 
tion of smooth stationary measures for nonequilibrium 
steady-states -the Sinai-Ruelle-Bowen (SRB) measures- 
the determination of which precisely rests on the dynamic 
partition function and the KS entropy 0. 

It must however be acknowledged that the determina- 
tion of any of these quantities has only been carried out 
for toy models of dynamical systems, such as the baker's 
map Q ■ Most of the efforts for physically relevant sys- 
tems have borne on the Lorentz gas 0, 0, 01 j an d more 
recently on hard-sphere systems in the dilute limit 7J. 
It is possible to relate discrete-time Markov processes to 
dynamical systems (see Q for a pedagogical account), 
yet very few examples going beyond the simple random 



walk have been investigated. More importantly, when the 
discrete time scale is sent to zero, infinities arise |8|, so 
that no viable definition of the dynamical partition func- 
tion in the physical limit of continuous time has hitherto 
been proposed. Continuous-time Markov processes are 
ubiquitous. Many systems governed by a Hamiltonian 
dynamics can be mapped, within some well-controlled 
approximation schemes, onto Markov processes. An ex- 
tensive activity in the physical modeling of complex sys- 
tems, from interacting spins to interacting gases, from 
avalanches in sandpiles to chemical reactions, relies on a 
continuous-time Markov description. In the present let- 
ter we construct, apparently for the first time in the liter- 
ature, the dynamical partition function for such Markov 
processes. We do not rely on a discrete-to-continuum 
limit and by working directly with continuous time, 
we obtain finite results. From this we shall show how 
to extract the KS entropy. We will further connect it, 
in the spirit of Gaspard to the Markov analog of a 
fluctuating entropy flow, as identified by Lebowitz and 
Spohn 10]. In order to exemplify our findings, we will 
then perform an exact calculation on the infinite-range 
(i.e. mean-field) Ising model, endowed with spin-flip dy- 
namics as introduced by Ruijgrok and Tjon |ll| . To the 
best of our knowledge, this is the first available result for 
a many-body interacting stochastic system. 

The outline of this letter is as follows: First we 
construct the dynamical partition function and the 
related topological pressure, which is identified as the 
large deviation function of a physical observable. From 
this we deduce an expression for the KS entropy. Then 
we establish a connection to the time-integrated entropy 
current. Finally, we explore the consequences of our 
formulation on three examples, a random walk with 
absorbing boundaries, a symmetric exclusion process, 
and an infinite-range Ising model, for which explicit and 
exact calculations are performed. 

We begin with a generic Markov process characterized 
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by transition rates W(C — > C) from configuration C to 
configuration C. The mean residence time in configu- 
ration C is l/r(C), with r(C) = £ c , c ')- This 
means in particular that the probability of hopping from 
configuration C after a time interval t to some other con- 
figuration between t and t + dt is r(C) exp(—r(C)t)dt. 
Among the allowed target configurations, the system 
hops to the particular configuration C with probability 
W(C — > C')/r{C). Various properties of the master equa- 
tion evolution operator W for the probability P(C,t) to 
be in state C at time t, such that, in matrix notation, 
dtP = WP, can be found in [l2j. We define the dynamic 
partition function Z(s, i) as the sum over all possible his- 
tories of the process over the interval [0, t] of the proba- 
bilities of the histories raised to the power (1 — s) [13|]- It 
is a matter of carefully applying the definition to realize 
that 



which verifies that 



Q+(t)-Q_(i) = ^ln 



n=l 



WjCn-l -» C n ) 

W(C n - C_i) 



*(*) (6) 



The quantity Qs(t) appearing in the rhs of © is precisely 
the time-integrated fluctuating entropy flow introduced 
by Lebowitz and Spohn |Ioj . which verifies, as t — > oo 
the Gallavotti-Cohen symmetry (e~ s ® s ) ~ (e - ' 1-8 ^ 3 }. 
A similar property for discrete time Markov processes, 
discussed at the level of the averages, was already noted 
by Gaspard 9]. In the same way as it can be seen that 
the entropy current 

■&(C) = 5>(C^C) mf|^| (7) 



Z(s,t) = 



E 

histories from 0— *t 



Probatory}) 1 s = ( e ~ s Q+( t )) (2) verifies d(Q s )/dt = (J S (C)), one can verify that 



where the observable Q+(t) depends on the sequence of 
states Co, Cfc occupied by the system over [0, t] through 
the relationship 



A— 1 



n=0 



W(C n -> C n+1 ) 
r(C n ) 



(3) 



It is necessary to describe the meaning of the brackets 
in J3J: they stand for an average over the number k 
of successive states occupied over [0,i], over the various 
configurations Cq, ...C^ visited by the system and, finally, 
over the time lapses that the system has been staying in 
each of those k states. All these quantities define a his- 
tory. Note that the topological or Ruelle pressure tp(s), 
canonically defined as ip(s) — limt_>oo jlnZ, is also the 
generating function of the cumulants of the physical ob- 
servable Q + . It is possible to write an evolution equation 
for P(C,Q+,t), the probability to be in state C at time 
t with Q+(t) = Q+, and a similar one for the related 
generating function J dQ + e~ 8 Q+P(C, Q + , t). The latter 
obeys a master-equation-like evolution with an operator 
W_|_ whose matrix elements are given by 

W+(C,C) = W(C -> C) l - s r(C') s - r(C)5 c ,c> (4) 

An important consequence is that tp( s ) is simply the 
largest eigenvalue of the evolution operator W+. The 
analog was already known Q for a discrete-time Markov 
process, but appears to be a new result for continuous- 
time processes. We stress that ifj is finite even though we 
are working in continuous time. Having reduced the com- 
putation of chaotic properties to a mere technical chal- 
lenge, we shall later present concrete physical examples 
in which the entire spectrum of W+ can be determined. 
We temporarily continue with abstract considerations by 
defining a similar quantity Q_ for the time-reversed pro- 
cess, 



)_(i) = E ln 



W(C„ 



r 



n=l 



(5) 



j±{c) = J2 W ( C -^ c ') ln 



W(C g C) 



(8) 



govern the evolutions of Q± according to d(Q±)/dt = 
(J±(C)), and they have the property that J+(C) — 
J_(C) = Js(C). The entropy current is usually propor- 
tional to a particle current 0, 0, 0] or to an energy 
current |T^ |. hence its physical meaning is clear. However 
neither of J± receives such a straightforward interpreta- 
tion. In equilibrium, that is when the rates W satisfy the 
detailed balance condition, both Q+ and Q— have the 
same large deviation function. It requires to relate Q + 
to quantities familiar in the dynamical systems theory to 
endow it with a more transparent physical meaning. The 
KS entropy, /iks , which is defined [2| as 



ft-Ks = J]] Probjhistory} lnProbjhistory} 



(9) 



hist. 



is also obtained, for a system without escape, from 

h KS =^'(0) = -(J+). (10) 

Therefore J+ appears to convey the physical meaning 
of an information content flow. Another quantity of 
interest is the topological entropy h top = -0(1), which 
counts the number of possible trajectories over [0,t]. 
We close our construction with a remark on SRB 
measures. Following the procedure outlined in Beck 
and Schlogl Q{, the stationary SRB measure can be 
obtained from a variational principle: in our case, this is 
the invariant measure P that renders the combination 
ipp (s) = h,Ks[P] — (1 — S )(Q+) p/t maximum when s = 0. 
The master equation possesses a unique stationary 
solution. The SRB measure can only be this stationary 
solution, for which the above combination becomes 
if>(s) = s /iks if s — > and one recovers the fact that 
%s = <0'(O). 
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We now present three simple applications of increas- 
ing complexity. Consider first a particle diffusing with 
diffusion constant D on a one-dimensional infinite line. 
We can illustrate on this example how a deterministic 
map allows to define for a stochastic system a Lyapunov 
exponent compatible with the thermodynamic formalism 
exposed above. Each jump of the particle can be de- 
scribed by a deterministic map as indicated in 0, 
This allows to define a corresponding Lyapunov expo- 
nent from the exponential divergence between initially 
close trajectories, averaging over the time lapses between 
jumps (which are randomly distributed as for the Markov 
process) . Here we find A = 2D In 2 which is related 
to the topological pressure ip(s) = 2D (2 s — 1) through 
V>'(0) = A. 

The same example can be used to illustrate the case 
of systems with escape. The particle now jumps on an 
infinite two-dimensional lattice slab of width £ with ab- 
sorbing boundaries. It is a trivial matter to diagonalize 
the corresponding W+ and to find its largest eigenvalue 
"0(s), which reads, for large £ 



t/>(s) = 4L>(4 S - I) - 4 s Dir 2 



(11) 



Hence the topological pressure verifies if>(0) = —7, where 
7 = Dtt 2 /£ 2 is the escape-rate of the particle. This re- 
sult is of course consistent with that established for the 
Lorentz gas and stands as yet another manifestation of 
the link between transport coefficients and the topologi- 
cal pressure |Tgj . 

Our second example is the Symmetric Exclusion Pro- 
cess, a gas of N mutually excluding particles diffusing on 
a one-dimensional lattice of L sites with periodic bound- 
ary conditions. Their hopping rate is denoted by D, and 
for simplicity we have confined our analysis to determin- 
ing the KS entropy. Denoting by p the average density, 
and by a(p) — 2p(l — p) (twice) its compressibility, we 
find that 

3 

h K s /D = La ln(Lo-) + a hx(La) + ^ + OQnL/L) (12) 

In order to establish this result we started from /jks — 
— (J+) and we exploited that the equilibrium state is per- 
fectly random [l!j- Our first comment on (|12|l is that 
/iks is not extensive in the system size. Second, the 
effect of the interaction is felt through a given combi- 
nation of the system size and of its compressibility. At 
half-filling the system shows its largest KS entropy, and 
this is likely related to the fact that also the number of al- 
lowed distinct microscopic states is maximum at p = 1/2. 
The dependence of /iks on p through a only arises from 
the particle-hole symmetry. 

As a third example, we consider an infinite range Ising 
model with Hamiltonian TL = — (X)<=i cr i) 2 /2-/V, endowed 
with spin-flip dynamics. Each spin <7j flips independently 
with a rate exp(— (3aiM/N), where (3 is the inverse tem- 
perature and where M = o~i is the total magnetiza- 
tion before the spin-flip. These flipping rates satisfy the 
detailed balance property with respect to the canonical 



distribution exp(— (37i). This system has the advantage 
of exhibiting a second-order phase transition, the generic 
effect of which on Lyapunov exponents has hitherto never 
been explicitly probed. Technically speaking, the master 
equation has an evolution operator that can be exactly 
diagonalized [Hi . We will now describe, skipping all tech- 
nical details |20|. our results for the topological pressure 
for this system. On the one hand we may follow the pro- 
cedure outlined in the first part of this letter, and identify 
a state C with a configuration {<7j} of the N spins. There 
are 2 N such configurations, and restricting ourselves to 
the high temperature phase (3 < 1, the exact diagonal- 
ization of W+ as defined in (0} leads to the eigenvalues 
(or Ruelle-Pollicott resonances) tp(s) — n<fi(s) with n G N, 
where 

ip(s) = {N s - 1)N + N S {1 - (1 - b)P) - <t>{s) 
<f>{s) = N S / 2 ^N S (1 + s/3(2 - /?)) - /3(2 - 0) (13) 

Again we remark that the topological pressure H13[) is not 
extensive in the number of spins, which also reflects on 
the KS entropy: 



h KS = N In N ■ 



2(1-/3) 



In TV 



P 2 



2(1-/3) 



(14) 



What can also be observed on i|14|) is that, due to the 
diverging susceptibility /3/(l — /3) at the critical point 
(3 = 1 our calculational technique ^l| ceases to be valid 
for systems with A^(l — j3) ~ 1. 

Another viewpoint, on the other hand, would have 
consisted in adopting a coarse-grained description of the 
same spin system and chosing to characterize its states 
by their total magnetization M (there are 2N such mag- 
netization states). Even though we are talking about the 
same system we are not talking about the same Markov 
process, hence the spectrum of W + differs from its pre- 
vious expression and the Ruelle pressure is now given by 



=JV(2*-l) + 2 s (l- S )(l-/3) 

- 2 s / 2 [2 S (1 - s(l - f3) 2 ) - /3(2 - 13)] 



1/2 (15) 



Now the corresponding KS entropy is extensive in the sys- 
tem size. The leading term in the system size expresses 
that, roughly speaking, the total magnetization under- 
goes a simple random walk. Interactions, and the effects 
of correlations, just as in l(T2"|) or (|T4l . are felt to the 
next order in the system size. This is where interesting 
physical features are to be looked for. 

We now summarize our findings. We have defined 
and constructed a dynamical partition function a la 
Ruelle, along with the related topological pressure for 
continuous-time Markov processes. The topological pres- 
sure appears to be the largest eigenvalue of some opera- 
tor, and is therefore a finite quantity, in spite of working 
within a continuous time formulation. A key quantity for 
characterizing dynamical randomness, the KS entropy, 
follows, along with the topological entropy and the sum 
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of positive Lyapunov exponents. We have put our defini- 
tions to work on a few simple examples, thus illustrating 
how straightforward the connection between topological 
pressure and escape rate is, and making a significant step 
towards realistic systems (with large number of degrees 
of freedom and interactions). Our formulation has the 
advantage of bringing concepts and quantities pertain- 
ing to other fields of studies -dynamical systems theory- 
back into the statistical mechanics community, and to 
which the available toolbox (simulations, mean-field ap- 
proximations, low density expansions, field theory, exact 
results, etc.) will apply. We insist that the observable 
Q+(t) from which the main dynamical quantities arise 
can be easily monitored when performing numerical sim- 
ulations. 

We believe that our formulation opens a series of new 
research routes. To begin with, it would be interest- 
ing to see, e.g. for a lattice gas, how the precise form 
of the interactions and of the microscopic dynamics af- 
fects the KS entropy, and possibly to help clarify ongo- 
ing debates [2l|. To what extent is the result obtained 
in (|12[1 universal? Next, it would be instructive to see if 
ergodicity-breaking features, which are suspected to char- 
acterize systems with glassy dynamics, can be identified 
on /iks- This could be first examined on simple systems 
like a particle diffusing in a random scenery, and then 
on a many-body glass forming lattice gas, such as the 
Kob- Andersen model. Even if no conceptual progress is 
to be expected from that, it would certainly be useful to 



specialize our approach to a widely used class of Markov 
processes, that which rests on Langevin equations (this 
could be achieved by resorting to the description used 
by Kurchan in 22]). This includes in particular fluctu- 
ating hydrodynamics, but also phcnomcnological models 
of chemical reactions, surface growth, turbulence, etc. It 
would also be interesting to see how the nonequilibrium 
nature can be traced back on the form and properties of 
the topological pressure. In that sense it would be useful 
to compare the dynamical entropies of a gas in equilib- 
rium and for the same gas driven out of equilibrium by 
the system boundaries or by a bulk field. Such a sys- 
tem develops long range correlations and, in the case of a 
bulk drive, nonequilibrium phase transitions may be ob- 
served [23, 13 i now these features affect the dynamical 
entropies deserves to be investigated. Finally, the link 
with dynamical system theory that we have presented 
in our first example still involves a stochastic part (the 
residence time in each visited state). It would be in- 
teresting to formalize a relationship between continuous- 
time Markov processes and fully deterministic flows. This 
identification, if possible, would perhaps allow for an in- 
terpretation of our KS entropy in terms of the sum of 
Lyapunov exponents for the corresponding flow. We hope 
that the present work will stand as a contribution towards 
welding communities somewhat impervious to each other. 
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